Overview: Cross Correlation
In some processes, a lag occurs between a change in a field value and the effect seen on the process target field. When the optimal lag is calculated, there is the highest correlation between the change in one field and the effect seen on another field.
Use this operation to automatically calculate the lag required, as a function of the highest correlation. The lag is calculated either over time or over the row indexes.
Trending the generated dataset will enable you to see the peak of highest correlation, which equates to the optimal lag required between cause fields and the affect target field.
Properties
Category: |
Transform |
Performance risk: |
Moderate performance risk. The size of the dataset will determine the speed of the operation. Note: To speed up the calculation of correlation over time, one suggestion is to resample the dataset with the intended step size as the new period. Correlation calculations using index can then be performed. |
Knowledge required: |
Working knowledge of the software. |
Effect on datasets
How many datasets are required to perform this operation? |
One |
Does it create a new dataset? |
Yes |
Can you reconfigure this operation? |
Yes |
Can you apply this operation to a locked dataset? |
Yes |
Does it modify the current dataset in any way? |
No |
Requirements
-
A dataset that contains at least two double or integer fields.
-
For time based lag calculations: a timestamp field is required.
Note: This operation does not support milliseconds. First use the Resample operation to remove milliseconds present in your data source.
Results
-
A dataset containing only a lag field for each selected field.
-
The values shown for each field are the correlation values at each step of the lag.
-
The values in this lag field can be trended - the highest points in the trend illustrate the greatest correlation.
-
The values in the generated table are not normalised.
-
If the lagged dataset does not overlap with reference dataset all results will be bad quality (NULL).
Related topics: